Reconciling detailed transaction feedback

ABSTRACT

Reconciling detailed transaction feedback by detecting a rating of a transaction, where the rating indicates a negative experience, mining the sentiment of words in feedback text that is included with or as part of the rating to detect whether the words indicate positive sentiment or negative sentiment, responsive to determining that the words in the feedback text indicate that the feedback text connotes a positive sentiment, adjusting the rating of the transaction. The mining may include testing words in the feedback text to detect whether the words indicate positive sentiment or negative sentiment by calculating a sentiment score.

TECHNICAL FIELD

The present application relates generally to the technical field ofuser-provided feedback that addresses the user experience in a givenonline transaction.

BACKGROUND

Ecommerce feedback is one of the core innovations that has enabled thesuccess of ecommerce. It not only showcases the performance ofsellers/buyers in the market place but it is also used to incentivizesellers in terms of fees and to measure trust and Bad Buyer Experiences(“BBE”) on an ecommerce site. But many customers do not fully understandwhat each numerical DSR rating, sometimes referred to herein as adetailed seller rating or “DSR” is supposed to mean, sometimes resultingin anomalous or inconsistent DSR's.

BRIEF DESCRIPTION OF THE DRAWINGS

Some embodiments are illustrated by way of example and not limitation inthe figures of the accompanying drawings in which:

FIG. 1 is a block diagram illustrating a network system, according toexample embodiments;

FIG. 2 is a block diagram of application servers that may form a part ofthe network system of FIG. 1, according to example embodiments;

FIG. 3 is an illustration of a sentiment extraction and reconciliationapplication according to an example embodiment;

FIG. 4 is an illustration of flowchart of sentiment mining according toan example embodiment;

FIG. 5 is an illustration of a flowchart of sentiment mining showingadjusting feedback about a seller according to an example embodiment;

FIGS. 6 through 8 are user interfaces that may be used in conjunctionwith the workflows of FIGS. 4 and 5 according to an example embodiment;

FIG. 9 is a flowchart illustrating a method according to an exampleembodiment; and

FIG. 10 is a block diagram of an example machine on which components ofvarious embodiments of the system may be executed.

DETAILED DESCRIPTION

Ecommerce feedback is one of the core innovations that has enabled thesuccess of ecommerce. It not only showcases the performance ofsellers/buyers in the market place but it is also used to incentivizesellers in terms of fees and to measure trust and Bad Buyer Experiences(“BBE”) on an ecommerce site. But many customers do not fully understandwhat each numerical DSR rating, sometimes referred to herein as adetailed seller rating or “DSR” is supposed to mean.

As one example, feedback may be given on some ecommerce sites by users,who may be a buyer, as a numeric scale from 1 to 5, where 1 may be lowsatisfaction and 5 may be high satisfaction. But the actual scale from 1to 5 (for example, what level of satisfaction is a 2, what level ofsatisfaction is a 3, what level of satisfaction is a 4, etc.) it is leftto user interpretation.

There are cases on ecommerce sites were buyers leave a numerical DSR=3rating (often considered a mediocre rating) but give glowing praises tothe seller in feedback text which may be filled in as part of thefeedback. For example the feedback text may say “Very nice.” “Thanks forthe quick delivery.” “Grandson loves it.” “I would buy from you again.”But these high praise comments do not comport with the above mediocreDSR rating of 3. In other cases, the foregoing high praise may evenresult in a DSR rating of 2, which is considered low. These examplesillustrate anomalous or even contradictory DSR ratings, for example, thenumeral DSR number does not match what one would expect, given theglowing praise of the feedback text, or vice versa. This may be due tothe buyer's misunderstanding or misinterpretation of the meaning of eachnumeral in the 1-5 scale. This may also be due to feedback influenced bybiases introduced by such things as cultural aspects and geographicallocation. For example, some cultures may habitually give low DSRs, evenfor good service.

Ecommerce site operators have noted that feedback text are often pithyand use very short text. Users may use various decorations using specialcharacters in their feedback text, to make the feedback look pretty.This text, decorations and special characters such as emoticons mayindicate the sentiment of emotion of the user when the feedback text isentered. Focusing on feedback phrase level emotion mining, or “sentimentmining,” has been found to be useful in understanding more precisely themeaning of a given numerical DSR rating. This emotion mining is notlimited to single sentiment in a feedback comment. There could bemultiple sentiments within the text of a feedback comment. For example,one part of the feedback text may be positive (“I really liked thejeans”) and another part of the feedback text may be negative (“but theshipment took too long to arrive”).

Stated another way, sentiment mining is an entire “package” ofdetermining positive sentiment and negative sentiment in feedback text.Sentiment mining may be implemented using not only opinion lexiconscontaining standard words but may also be implemented from human ormachine learned sentiments from negative feedback provided to theecommerce system over time.

For example, negative modifiers may also be taken into account which,when detected or “mined” from feedback text may be an indication of abad buying experience (“BBE”). This allows detection of feedback textwhere the user is not pleased, for example by saying I am not happy.Examples of negative modifiers may be:

TABLE 1 not none doesn't doesnt didnt didn't wouldn't wouldnt wont won'tshouldnt shouldn't isn't isnt wasnt wasn't aren't arent werent weren'taint ain't couldn't couldnt can't cant

In addition, certain words used in feedback text may be found, overtime, to have specific negative connotations. Examples of words that maybe determined to have specific negative connotations may be:

TABLE 2 different differ counterfeit fake cheap issue issues ignoreignored broken late

Also, certain contrasting conjunctions may be mined and taken intoaccount as a part of sentiment mining. Contrasting conjunctions maydetect feedback in which the user makes a positive statement about theseller but also contrasts the positive statement with another, perhapsnegative, comment in the feedback text.

Examples of contrasting conjunctions may be:

TABLE 3 but although instead never yet than however

Positive sentiment may be mined using a positive lexicon. For example,positive words such as those in Table 4 below may be used for thispurpose. Examples of words of a positive lexicon are seen in the tablebelow. Many additional positive lexicon words will be apparent to thoseof ordinary skill in the art.

a+ abound abounds abundance abundant accessable accessible acclaimacclaimed acclamation accolade accolades accommodative accomodativeaccomplish accomplished accomplishment accomplishments accurateaccurately achievable achievement achievements achievible acumenadaptable adaptive adequate adjustable admirable admirably admirationadmire admirer admiring admiringly adorable adore adored adorer adoringadoringly adroit adroitly adulate adulation adulatory advanced advantageadvantageous advantageously advantages adventuresome adventurousadvocate advocated advocates affability affable affably affectationaffection affectionate affinity affirm affirmation affirmative affluenceaffluent afford affordable affordably afordable agile agilely agilityagreeable agreeableness agreeably all-around alluring alluringlyaltruistic altruistically amaze amazed amazement amazes amazingamazingly ambitious ambitiously ameliorate amenable amenity amiabilityamiabily amiable amicability amicable amicably amity ample amply amuseamusing amusingly angel angelic apotheosis appeal appealing applaudappreciable appreciate appreciated appreciates appreciativeappreciatively appropriate approval approve ardent ardently ardorarticulate aspiration aspirations aspire assurance assurances assureassuredly assuring astonish astonished astonishing astonishinglyastonishment astound astounded astounding astoundingly astutelyattentive attraction attractive attractively attune audible audiblyauspicious authentic authoritative autonomous available aver avid avidlyaward awarded awards awe awed awesome awesomely awesomeness awestruckawsome backbone balanced bargain beauteous beautiful beautifulllybeautifully beautify beauty beckon beckoned beckoning beckons believablebelieveable beloved benefactor beneficent beneficial beneficiallybeneficiary benefit benefits benevolence benevolent benifits bestbest-known best-performing best-selling better better-knownbetter-than-expected beutifully blameless bless blessing bliss blissfulblissfully blithe blockbuster bloom blossom bolster bonny bonus bonusesboom booming boost boundless bountiful brainiest brainy brand-new bravebravery bravo breakthrough breakthroughs breathlessness breathtakingbreathtakingly breeze bright brighten brighter brightest brilliancebrilliances brilliant brilliantly brisk brotherly bullish buoyant cajolecalm calming calmness capability capable capably captivate captivatingcarefree cashback cashbacks catchy celebrate celebrated celebrationcelebratory champ champion

In addition, colloquialisms that may be specific to a given ecommercesystem, as well as slang and sarcasms, may be detected in feedback textand used in sentiment mining. Further, there may be cases taken intoaccount in the sentiment mining described where words in the feedbacktext that typically indicate negative sentiment bearing may not, infact, be negative. For example, in the term “excellent worn-out jeans,”the term “worn-out” is usually a negative term. However, if the productwere so called “destroyed jeans” (that is, jeans manufactured to befaded or torn to give the appearance of having been worn and washedseveral times) the term “worn-out” could be considered a positivesentiment.

The following disclosure addresses these problems by extracting thesentiment from the feedback text using the components discussed abovefor sentiment mining, and then reconciling the sentiment with thenumerical DSR rating. The DSRs may then be normalized by eliminatingindividual biases. This may be done for individual buyers sometimesreferred to as being done on a “per buyer” basis.

In an embodiment, offline data mining preparation may include miningfrom one or more historical ecommerce transaction logs. Known specificnegative words may be mined from the text of feedback text from thetransaction log. Phrase-level emotion or sentiment language may be minedfrom the feedback text to detect negative sentiment. The averagenumerical DSRs may be analyzed, in one embodiment on a continual basis,for each geography of the ecommerce system, each category of listeditem, each average selling price, and the like. These average DSRs maylater be used in DSR normalization to be discussed below.

Sentiment may then be mined from the feedback text and compared with therespective DSRs provided by the user. The discrepancy between the minednegative sentiments and the respective DSRs may be reconciled in theflow. Reconciliation may be done either real-time online when the buyeris leaving contradictory feedback or in an offline mode where separateoffline survey could be conducted to derive the weights the current DSRrating should carry when it differs from feedback text. Weights given tothe DSRs may then be auto-adjusted to reflect the disagreement betweenthe numerical value of the DSRs and the mined sentiments from the DSRs.DSR ratings may be computed by taking a simple average of all the DSRsgiven to a seller by buyers of his items. A way of deriving at theaggregate DSR ratings could be envisioned in which weight for theratings, where the feedback text and rating do not appear consistent, islowered. The exact weight could be derived in a data driven way. Forexample, in an embodiment, a selective offline survey may be addressedto buyers who left numerical DSRs that did not match the detectedsentiment in the feedback text that the buyers left with their DSRs. Theanswers to the survey may be used in DSR normalization.

FIG. 1 is a network diagram depicting a network system 100, according toone embodiment, having a client-server architecture configured forexchanging data over a network. For example, the network system 100 mayinclude a network-based publisher 102 where clients may communicate andexchange data within the network system 100. The data may pertain tovarious functions (e.g., online item purchases) and aspects e.g.,managing content) associated with the network system 100 and its users.Although illustrated herein as a client-server architecture as anexample, other embodiments may include other network architectures, suchas a peer-to-peer or distributed network environment.

A data exchange platform, in an example form of a network-basedpublisher 102, may provide server-side functionality, via a network 104(e.g., the Internet, wireless network, cellular network, or a Wide AreaNetwork (WAN)) to one or more clients. The one or more clients mayinclude users that utilize the network system 100 and more specifically,the network-based publisher 102, to exchange data over the network 104.These transactions may include transmitting, receiving (communicating)and processing data to, from, and regarding content and users of thenetwork system 100. The data may include, but are not limited to,content and user data such as feedback data; user profiles; userattributes; product attributes; product and service reviews; product,service, manufacture, and vendor recommendations and identifiers; socialnetwork commentary, product and service listings associated with buyersand sellers; auction bids; and transaction data, among other things.

In various embodiments, the data exchanges within the network system 100may be dependent upon user-selected functions available through one ormore client or user interfaces (UIs). The UIs may be associated with aclient device, such as a client device 110 using a web client 106. Theweb client 106 may be in communication with the network-based publisher102 via a web server 116. The UIs may also be associated with a clientdevice 112 using a programmatic client 108, such as a clientapplication. It can be appreciated in various embodiments the clientdevices 110, 112 may be associated with a buyer, a seller, a third partyelectronic commerce platform, a payment service provider, or a shippingservice provider, each in communication with the network-based publisher102 and optionally each other. The buyers and sellers may be any one ofindividuals, merchants, or service providers, among other things. Theclient devices 110 and 112 may comprise a mobile phone, desktopcomputer, laptop, or any other communication device that a user may useto access the network-based publisher 102.

Turning specifically to the network-based publisher 102, an applicationprogram interface (API) server 114 and a web server 116 are coupled to,and provide programmatic and web interfaces respectively to, one or moreapplication servers 118. The application servers 118 host one or morepublication application(s) of publication system 120 and one or morepayment systems 122. The application server(s) 118 are, in turn, shownto be coupled to one or more database server(s) 124 that facilitateaccess to one or more database(s) 126.

In one embodiment, the web server 116 and the API server 114 communicateand receive data pertaining to products, listings, transactions, socialnetwork commentary and feedback, among other things, via various userinput tools. For example, the web server 116 may send and receive datato and from a toolbar or webpage on a browser application (e.g., webclient 106) operating on a client device (e.g., client device 110). TheAPI server 114 may send and receive data to and from an application(e.g., client application 108) running on another client device (e.g.,client device 112).

The publication system 120 publishes content on a network (e.g., theInternet). As such, the publication system 120 provides a number ofpublication and marketplace functions and services to users that accessthe network-based publisher 102. For example, the publicationapplication(s) of publication system 120 may provide a number ofservices and functions to users for listing goods and/or services forsale, facilitating transactions, and reviewing and providing feedbackabout transactions and associated users. Additionally, the publicationapplication(s) of publication system 120 may track and store data andmetadata relating to products, listings, transactions, and userinteraction with the network-based publisher 102. The publicationapplication(s) of publication system 120 may aggregate the tracked dataand metadata to perform data mining to identify trends or patterns inthe data. While the publication system 120 may be discussed in terms ofa marketplace environment, it may be noted that the publication system120 may be associated with a non-marketplace environment.

The payment system 122 provides a number of payment services andfunctions to users. The payment system 122 allows users to accumulatevalue (e.g., in a commercial currency, such as the U.S. dollar, or aproprietary currency, such as “points”) in accounts, and then later toredeem the accumulated value for products (e.g., goods or services) thatare made available via the publication system 120. The payment system122 also facilitates payments from a payment mechanism (e.g., a bankaccount, PayPal account, or credit card) for purchases of items via thenetwork-based marketplace. While the publication system 120 and thepayment system 122 are shown in FIG. 1 to both form part of thenetwork-based publisher 102, it will be appreciated that, in alternativeembodiments, the payment system 122 may form part of a payment servicethat may be separate and distinct from the network-based publisher 102.

Application Server(s)

FIG. 2 illustrates a block diagram showing applications of applicationserver(s) that are part of the network system 100, in an exampleembodiment. In this embodiment, the publication system 120, and thepayment system 120 may be hosted by the application server(s) 118 of thenetwork system 100. The publication system 120 and the payment system132 may be hosted on dedicated or shared server machines (not shown)that are communicatively coupled to enable communications between servermachines. The applications themselves may be communicatively coupled(e.g., via appropriate interfaces) to each other and to various datasources, so as to allow information to be passed between theapplications or so as to allow the applications to share and accesscommon data.

The publication system 120 are shown to include at least one or moreauction application(s) 212 which support auction-format listing andprice setting mechanisms (e.g., English, Dutch, Vickrey, Chinese,Double, Reverse auctions etc.). The auction application(s) 212 may alsoprovide a number of features in support of such auction-format listings,such as a reserve price feature whereby a seller may specify a reserveprice in connection with a listing and a proxy-bidding feature whereby abidder may invoke automated proxy bidding. The auction-format offer inany format may be published in any virtual or physical marketplacemedium and may be considered the point of sale for the commercetransaction between a seller and a buyer (or two users).

One or more fixed-price application(s) 214 support fixed-price listingformats (e.g., the traditional classified advertisement-type listing ora catalogue listing) and buyout-type listings. Specifically, buyout-typelistings (e.g., including the Buy-It-Now® (BIN) technology developed byeBay Inc., of San Jose, Calif.) may be offered in conjunction withauction-format listings, and allow a buyer to purchase goods orservices, which are also being offered for sale via an auction, for afixed-price that may be typically higher than the starting price of theauction.

The application(s) of the application server(s) 118 may include one ormore store application(s) 216 that allow a seller to group listingswithin a “virtual” store. The virtual store may be branded and otherwisepersonalized by and for the seller. Such a virtual store may also offerpromotions, incentives and features that are specific and personalizedto a relevant seller.

Navigation of the online marketplace may be facilitated by one or morenavigation application(s) 220. For example, a search application (as anexample of a navigation application) may enable key word searches oflistings published via the network-based publisher 102. A browseapplication may allow users to browse various category, catalogue, orinventory data structures according to which listings may be classifiedwithin the network-based publisher 102. Various other navigationapplications may be provided to supplement the search and browsingapplications.

Merchandizing application(s) 222 support various merchandising functionsthat are made available to sellers to enable sellers to increase salesvia the network-based publisher 102. The merchandizing application(s)222 also operate the various merchandising features that may be invokedby sellers, and may monitor and track the success of merchandisingstrategies employed by sellers.

Personalization application(s) 230 allow users of the network-basedpublisher 102 to personalize various aspects of their interactions withthe network-based publisher 102. For example, a user may, utilizing anappropriate personalization application 230, create a personalizedreference page at which information regarding transactions to which theuser may be (or has been) a party may be viewed. Further, thepersonalization application(s) 230 may enable a third party topersonalize products and other aspects of their interactions with thenetwork-based publisher 102 and other parties, or to provide otherinformation, such as relevant business information about themselves.

The publication system 120 may include one or more internationalizationapplication(s) 232. In one embodiment, the network-based publisher 102may support a number of marketplaces that are customized, for example,for specific geographic regions. A version of the network-basedpublisher 102 may be customized for the United Kingdom, whereas anotherversion of the network-based publisher 102 may be customized for theUnited States. Each of these versions may operate as an independentmarketplace, or may be customized (or internationalized) presentationsof a common underlying marketplace. The network-based publisher 102 mayaccordingly include a number of internationalization application(s) 232that customize information (and/or the presentation of information) bythe network-based publisher 102 according to predetermined criteria(e.g., geographic, demographic or marketplace criteria). For example,the internationalization application(s) 232 may be used to support thecustomization of information for a number of regional websites that areoperated by the network-based publisher 102 and that are accessible viarespective web servers.

Reputation application(s) 234 allow users that transact, utilizing thenetwork-based publisher 102, to establish, build and maintainreputations, which may be made available and published to potentialtrading partners. Consider that where, for example, the network-basedpublisher 102 supports person-to-person trading, users may otherwisehave no history or other reference information whereby thetrustworthiness and credibility of potential trading partners may beassessed. The reputation application(s) 234 allow a user, for examplethrough feedback provided by other transaction partners, to establish areputation within the network-based publisher 102 over time. Otherpotential trading partners may then reference such a reputation for thepurposes of assessing credibility and trustworthiness.

In order to make listings, available via the network-based publisher102, as visually informing and attractive as possible, the publicationsystem 120 may include one or more imaging application(s) 236 utilizingwhich users may upload images for inclusion within listings. An imagingapplication 236 also operates to incorporate images within viewedlistings. The imaging application(s) 236 may also support one or morepromotional features, such as image galleries that are presented topotential buyers. For example, sellers may generally pay an additionalfee to have an image included within a gallery of images for promoteditems.

The publication system 120 may include one or more offer creationapplication(s) 238. The offer creation application(s) 238 allow sellersconveniently to author products pertaining to goods or services thatthey wish to transact via the network-based publisher 102. Offermanagement application(s) 240 allow sellers to manage offers, such asgoods, services, or donation opportunities. Specifically, where aparticular seller has authored and/or published a large number ofproducts, the management of such products may present a challenge. Theoffer management application(s) 240 provide a number of features (e.g.,auto-reproduct, inventory level monitors, etc.) to assist the seller inmanaging such products. One or more post-offer management application(s)242 also assist sellers with a number of activities that typically occurpost-offer. For example, upon completion of an auction facilitated byone or more auction application(s) 212, a seller may wish to leavefeedback regarding a particular buyer. To this end, a post-offermanagement application 242 may provide an interface to one or morereputation application(s) 234, so as to allow the seller conveniently toprovide feedback regarding multiple buyers to the reputationapplication(s) 234.

The dispute resolution application(s) 246 may provide mechanisms wherebydisputes arising between transacting parties may be resolved. Forexample, the dispute resolution application(s) 246 may provide guidedprocedures whereby the parties are guided through a number of steps inan attempt to settle a dispute. In the event that the dispute cannot besettled via the guided procedures, the dispute may be escalated to amediator or arbitrator.

The fraud prevention application(s) 248 may implement various frauddetection and prevention mechanisms to reduce the occurrence of fraudwithin the network-based publisher 102. The fraud preventionapplication(s) may prevent fraud with respect to the third party and/orthe client user in relation to any part of the request, payment,information flows and/or request fulfillment. Fraud may occur withrespect to unauthorized use of financial instruments, non-delivery ofgoods, and abuse of personal information.

Authentication application(s) 250 may verify the identity of a user, andmay be used in conjunction with the fraud prevention application(s) 248.The user may be requested to submit verification of identity, anidentifier upon making the purchase request, for example. Verificationmay be made by a code entered by the user, a cookie retrieved from thedevice, a phone number/identification pair, a username/password pair,handwriting, and/or biometric methods, such as voice data, face data,iris data, finger print data, and hand data. In some embodiments, theuser may not be permitted to login without appropriate authentication.The system (e.g., the FSP) may automatically recognize the user, basedupon the particular network-based device used and a retrieved cookie,for example.

The network-based publisher 102 itself, or one or more parties thattransact via the network-based publisher 102, may operate loyaltyprograms and other types of promotions that are supported by one or moreloyalty/promotions application(s) 254. For example, a buyer/client usermay earn loyalty or promotions points for each transaction establishedand/or concluded with a particular seller/third party, and may beoffered a reward for which accumulated loyalty points can be redeemed.

The application server(s) 118 may include messaging application(s) 256.The messaging application(s) 256 are responsible for the generation anddelivery of messages to client users and third parties of thenetwork-based publisher 102. Information in these messages may bepertinent to services offered by, and activities performed via, thepayment system 120. Such messages, for example, advise client usersregarding the status of products (e.g., providing “out of stock” or“outbid” notices to client users) or payment status (e.g., providinginvoice for payment, Notification of a Payment Received, deliverystatus, invoice notices). Third parties may be notified of a productorder, payment confirmation and/or shipment information. Respectivemessaging application(s) 256 may utilize any one of a number of messagedelivery networks and platforms to deliver messages to users. Forexample, messaging application(s) 256 may deliver electronic mail(email), instant message (IM), Short Message Service (SMS), text,facsimile, or voice (e.g., Voice over IP (VoIP)) messages via the wired(e.g., the Internet), Plain Old Telephone Service (POTS), or wireless(e.g., mobile, cellular, WiFi, WiMAX) networks.

The payment system 120 may include one or more payment processingapplication(s) 258. The payment processing application(s) 258 mayreceive electronic invoices from the merchants and may receive paymentsassociated with the electronic invoices. The payment system 120 may alsomake use of functions performed by some applications included in thepublication system 120.

The publication system 120 may include one or more sentiment extractionand reconciliation applications 260.

FIG. 3 is an illustration of a sentiment extraction and reconciliationapplication 300, which is a further description of item 260 of FIG. 2according to an example embodiment. In one embodiment sentimentextraction and reconciliation application 300 of FIG. 3 may operateoffline if desired and may mine historical transaction logs of anecommerce system. This sentiment mining may be done on a continual basisif desired, or mined from time to time. Item 300 may include positivesentiment mining tool 310 which may be an application for detectingpositive words such as those in Table 1, or other words learned toindicate positive sentiment.

Item 300 may also include negative sentiment mining tool 320 which maybe an application for detecting negative modifiers such as those ofTable 1, above, detecting words having negative connotations, such asthose of Table 2, above, and detecting contrasting conjunctions such asthose in Table 3, above; and also for detecting words learned toindicate negative sentiment.

Phrase level sentiment mining tool 330 may be an application fordetecting phrase-level sentiment. The comments left by Buyers forsellers are pithy and can cover various aspects of the transaction in anot grammatically well formed manner. For example, a feedback commentcould be “Got it fast, received in great condition”. Here buyer isexpressing his feelings about ship time as well as item condition. Thesetwo comment segments may be separated using the obvious markers likecomma, semi colon, colon, multiple spaces or tabs, periods and otherpunctuation marks. When no punctuation marks are available,probabilistic techniques like Hidden Markov Models (HMM) could be usedto break one feedback comment into comment segments. Once commentsegments are identified, sentiment mining could be done on eachphrase/segment to extract positive/negative feelings of the buyertowards a specific aspect of the transaction.

Average numerical detailed seller rating (DSR) analysis tool 340 may byan application for analyzing the DSR of users in the transaction logs.Item 340 may analyze, on a continual basis if desired, or periodically,may analyze average DSRs left selected geographies, item listingcategory, average selling price, and/or other desirable indicators todetermine whether there is any feedback bias in any of the foregoingselections, tier example, by culture, country, or otherwise. In oneembodiment the average DSR rating given by a user in the past year forpositive feedback transactions may be detected. Then that average may beapplied to current DSRs to determine true BBEs in those DSRs.

Selective offline survey 350 may be an application for surveying buyerswhose DSRs were detected in the above analysis not to match thesentiment in the respective buyer's sentiment in the feedback texts.This clarification survey may reconcile the meaning of the feedback textand a DSR. In one embodiment the results of this clarification surveymay be used to adjust the DSR or the feedback text.

Detailed seller rating normalization tool 360 may then normalize sellerratings according to an auto-adjustment procedure. The tool may find theaverage IAD detailed seller rating that a buyer tends to leave. Thisaverage is computed for each buyer's transactions in the past year forwhich positive feedback was left and no dispute was filed by the buyer.This average value for each buyer can be used to normalize his DSRs.

FIG. 4 is an illustration of flowchart, or work flow, of sentimentmining according to an example embodiment. At 410 a user, here a buyer,provides feedback text and a numerical detailed seller rating. This maybe seen in further detail on the user interface 410A of FIG. 6 in whichthe seller has the opportunity of rating the transaction at 411A, ofleaving feedback text at 413A, and of providing one or more numericalDSRs about the purchase at 415A. In this example embodiment the scalemay indicate 1-5 reading left to right, where 1 is low and 5 is high.The detailed seller rating normalization 360 tool finds the average DSRthat a buyer tends to leave. This average is computed for each buyer'stransactions in the past year for which positive feedback was left andno dispute was filed by the buyer. This average value for each buyer canbe used to normalize his DSRs.

At 420 whether the DSR is less than or equal to a 3 is checked for bywell-known comparison methods. As discussed above, a rating of 3 isconsidered in the present instance as being a mediocre rating, soratings of 3 or less are used for checking purposes. A reason forchecking for ratings of 3 or less is that as part of analyzing feedbacktext to understand the reasons why negative/neutral DSR may be provided,initial analysis began by looking into feedback text, Inasmuch as IADDSRs (ratings of the truth of the “Item As Described”) form a goodportion of BBEs, analysis initially focused on IAD DSRs of less than orequal to 3 (which, while “mediocre” may for business purposes beconsidered an indication of a BBE) and on the corresponding BBEs.Analyzing the feedback text of the few of the IAD DSRs that were lessthan or equal to 3, indicated that there were a good amount of purelypositive feedback text in IAD DSRs that numerically indicated a BBE.This appears to be a contradiction. This determination may be performedusing specific positive word mining tool 310, negative sentiment miningtool 320, and phase level sentiment mining tool 330 as discussed furtherbelow. At 430 a determination of whether there is positive feedback inthe feedback text is made. If there is no positive feedback for an IADof less than or equal to 3, the feedback is considerednon-contradictory, and it is stored in database 440 for subsequent use.A positive sentiment word lexicon such as that in Table 4 may be used tosee if there are positive sentiments in the Feedback. Using this alongwith negative modifier lists, and contrasting conjunctions, discussedabove, allows determining if the feedback has only positive sentiment.

If, on the other hand, there is positive feedback text in an IAD of lessthan or equal to 3, an algorithm, discussed below, is run at 450 tocheck if there is any negative sentiment in the feedback text. Again,this may be accomplished at least in part by phrase level sentimentmining tool 330 discussed above in connection with FIG. 3. If thedecision at 460 is yes (i.e., there is negative sentiment in thefeedback text), then the feedback text and the low IAD DSR may beconsidered to be consistent. An example of this is seen at the UI 470Aof FIG. 7. In this case, the feedback text (“But received late”) wasnegative, and the IAD DSR at 471 was 2, so the negative feedback textand the low IAD are thought to be consistent, and the IAD DSR is storedin database 470 in FIG. 4 for future use.

If decision 460 detects that there is no negative sentiment feedbackthen the feedback text and the IAD DSR of 2 may be consideredinconsistent and the DSR may be confirmed with the buyer as at 480 sothat discrepancies between the feedback text and the IAD numericalrating are reconciled. This inconsistency may be seen in FIG. 8 wherethe transaction is rated “positive,” (i.e., the feedback text is “A++++seller. Thanks.”) but the IAD is a 2, as at 481.

FIG. 5 is an illustration of a flowchart of sentiment mining showingadjusting feedback about a seller according to an example embodiment.FIG. 5 is much the same as FIG. 4 so only the differences will bediscussed here. These differences are that if the decision 560 from thealgorithm that is run at 550 (which may be the same algorithm as at step450 of FIG. 4) is that there is no negative sentiment expressed in thefeedback text, then the low IAD of less than or equal to 3, and the lackof negative sentiment in the feedback text, appear inconsistent. Thismay be reconciled as indicated at 580 by assigning a lower weight factorto the DSR because of the apparent inconsistency. The DSR weighting maybe based on buyer average feedback and other biases. As discussed above,these biases may be geographies, item listing category, average sellingprice, and/or other desirable indicators to determine whether there isany feedback bias in any of the foregoing selections, for example, byculture, country, or otherwise. The adjustment may ales be based onaverage feedback, for example, whether the DSR is significantlydifferent from the DSRs of the user as averaged over a previous periodof time as discussed above.

FIG. 9 is a flowchart illustrating a method 900 according to an exampleembodiment. In sentiment mining of feedback texts, the particularfeedback text may be split into individual words as at 910. Thenon-alphanumeric characters may be removed from each word as at 920. At930 a determination is made as to whether the word under test in theparticular feedback text appears positive or negative. The test forpositive sentiment may be made using positive sentiment mining tool 310testing for words such as those in Table 4, and words learned toindicate positive sentiment. The test for negatives sentiment may bemade using negative sentiment mining tool 320 testing for words such asthose in Tables 1, 2, and 3, and words learned to indicate negativesentiment. If the test at 930 determines that the word under testappears positive, then at 980 a decision is made as to whether there areany negative words in the particular feedback text of which the wordunder test is a part. If Yes, then the feedback text may be flagged asnegative as at 970. If No, then a positive word counter for theparticular feedback text may be incremented. The determination ofwhether there are any negative words in the particular feedback text (intest 980) may be implemented by testing to determine whether all wordsin the feedback text have been tested. For example a loop (not shown)could be embodied that determines whether a negative word has beendetected in the particular feedback text and, if not, testing to see ifthe final word of the particular feedback text has been tested fornegative. If the final word in a particular feedback text has beentested without encountering a negative word, this would be an indicatorto increment the positive word counter as at 990, and as discussedabove.

With continued reference to FIG. 9, if a word appears negative at 930then a test is made at 940 to determine whether the word is in thenegative modifier list of Table 1. If the word is in the negativemodifier list, the feedback text is flagged as negative at 970, whichindicates that the feedback text indicates negative sentiment.

When the test for the particular feedback text is completed (which maybe tested by implementation of a loop similar to that discussed abovefor FIG. 9) the negative word counter and the positive word counterindicate that the feedback test is are totaled and the ratio ofpositive/(positive+negative) may be used to determine if the feedback issignificantly positive. Stated another way, the total and ratio may beconsidered a calculation of a score or threshold which, if it exceeds apredetermined amount, indicates that the feedback text indicates anoverall positive sentiment. The thresholds of positiveness of a feedbackcomment may be very high but could be adjusted as needed. In oneembodiment, the score or threshold is calculated if the test at 940 doesnot result in the feedback text being flagged as negative as at 970.

When the test for the particular feedback text is completed (which maybe tested by implementation of a loop similar to that discussed abovefor FIG. 9) the negative word counter and the positive word counter aretotaled and the ratio of positive/(positive+negative) may be used todetermine if the feedback is significantly positive. The thresholds ofpositiveness of a feedback text may be very high but could be adjustedas needed.

Modules, Components, and Logic

Additionally, certain embodiments described herein may be implemented aslogic or a number of modules, engines, components, or mechanisms. Amodule, engine, logic, component, or mechanism (collectively referred toas a “module”) may be a tangible unit capable of performing certainoperations and configured or arranged in a certain manner. In certainexample embodiments, one or more computer systems (e.g., a standalone,client, or server computer system) or one or more components of acomputer system (e.g., a processor or a group of processors) may beconfigured by software (e.g., an application or application portion) orfirmware (note that software and firmware can generally be usedinterchangeably herein as may be known by a skilled artisan) as a modulethat operates to perform certain operations described herein.

In various embodiments, a module may be implemented mechanically orelectronically. For example, a module may comprise dedicated circuitryor logic that may be permanently configured (e.g., within aspecial-purpose processor, application specific integrated circuit(ASIC), or array) to perform certain operations. A module may alsocomprise programmable logic or circuitry (e.g., as encompassed within ageneral-purpose processor or other programmable processor) that may betemporarily configured by software or firmware to perform certainoperations. It will be appreciated that a decision to implement a modulemechanically, in dedicated and permanently configured circuitry, or intemporarily configured circuitry (e.g., configured by software) may bedriven by, for example, cost, time, energy-usage, and package sizeconsiderations.

Accordingly, the term “module” should be understood to encompass atangible entity, be that an entity that may be physically constructed,permanently configured (e.g., hardwired), or temporarily configured(e.g., programmed) to operate in a certain manner or to perform certainoperations described herein. Considering embodiments in which modules orcomponents are temporarily configured (e.g., programmed), each of themodules or components need not be configured or instantiated at any oneinstance in time. For example, where the modules or components comprisea general-purpose processor configured using software, thegeneral-purpose processor may be configured as respective differentmodules at different times. Software may accordingly configure theprocessor to constitute a particular module at one instance of time andto constitute a different module at a different instance of time.

Modules can provide information to, and receive information from, othermodules. Accordingly, the described modules may be regarded as beingcommunicatively coupled. Where multiples of such modules existcontemporaneously, communications may be achieved through signaltransmission (e.g., over appropriate circuits and buses) that connectthe modules. In embodiments in which multiple modules are configured orinstantiated at different times, communications between such modules maybe achieved, for example, through the storage and retrieval ofinformation in memory structures to which the multiple modules haveaccess. For example, one module may perform an operation and store theoutput of that operation in a memory device to which it may becommunicatively coupled. A further module may then, at a later time,access the memory device to retrieve and process the stored output.Modules may also initiate communications with input or output devicesand can operate on a resource (e.g., a collection of information).

Example Machine Architecture and Machine-Readable Storage Medium

With reference to FIG. 10 an example embodiment extends to a machine inthe example form of a computer system 1000 within which instructions forcausing the machine to perform any one or more of the methodologiesdiscussed herein may be executed. In alternative example embodiments,the machine operates as a standalone device or may be connected (e.g.,networked) to other machines. In a networked deployment, the machine mayoperate in the capacity of a server or a client machine in server-clientnetwork environment, or as a peer machine in a peer-to-peer (ordistributed) network environment. The machine may be a personal computer(PC), a tablet PC, a set-top box (SIB), a Personal Digital Assistant(PDA), a cellular telephone, a web appliance, a network router, a switchor bridge, or any machine capable of executing instructions (sequentialor otherwise) that specify actions to be taken by that machine. Further,while only a single machine may be illustrated, the term “machine” shallalso be taken to include any collection of machines that individually orjointly execute a set (or multiple sets) of instructions to perform anyone or more of the methodologies discussed herein.

The example computer system 1000 may include a processor 1002 (e.g., acentral processing unit (CPU), a graphics processing unit (GPU) orboth), a main memory 1004 and a static memory 1006, which communicatewith each other via a bus 1007. The computer system 1000 may furtherinclude a video display unit 1010 (e.g., a liquid crystal display (LCD)or a cathode ray tube (CRT)). In example embodiments, the computersystem 1000 also includes one or more of an alpha-numeric input device1012 (e.g., a keyboard), a user interface (UI) navigation device orcursor control device 1014 (e.g., a mousse), a disk drive unit 1016, asignal generation device 1018 (e.g., a speaker), and a network interfacedevice 1020.

Machine-Readable Medium

The disk drive unit 1016 includes a machine-readable storage medium 1022on which may be stored one or more sets of instructions 1024 and datastructures (e.g., software instructions) embodying or used by any one ormore of the methodologies or functions described herein. Theinstructions 1024 may also reside, completely or at least partially,within the main memory 1004 or within the processor 1002 duringexecution thereof by the computer system 1000, with the main memory 1004and the processor 1002 also constituting machine-readable media.

While the machine-readable storage medium 1022 may be shown in anexample embodiment to be a single medium, the term “machine-readablestorage medium” may include a single storage medium or multiple storagemedia (e.g., a centralized or distributed database, or associated cachesand servers) that store the one or more instructions. The term“machine-readable storage medium” shall also be taken to include anytangible medium that may be capable of storing, encoding, or carryinginstructions for execution by the machine and that cause the machine toperform any one or more of the methodologies of embodiments of thepresent application, or that may be capable of storing, encoding, orcarrying data structures used by or associated with such instructions.The term “machine-readable storage medium” shall accordingly be taken toinclude, but not be limited to, solid-state memories and optical andmagnetic media. Specific examples of machine-readable storage mediainclude non-volatile memory, including by way of example semiconductormemory devices (e.g., Erasable Programmable Read-Only Memory (EPROM),Electrically Erasable Programmable Read-Only Memory (EEPROM), and flashmemory devices); magnetic disks such as internal hard disks andremovable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks.

Transmission Medium

The instructions 1024 may further be transmitted or received over acommunications network 1026 using a transmission medium via the networkinterface device 1020 and utilizing any one of a number of well-knowntransfer protocols (e.g., Hypertext Transfer Protocol (HTTP)). Examplesof communication networks include a local area network (LAN), a widearea network (WAN), the Internet, mobile telephone networks, Plain OldTelephone Service (POTS) networks, and wireless data networks (e.g.,WiFi and WiMax networks). The term “transmission medium” shalt be takento include any intangible medium that may be capable of storing,encoding, or carrying instructions for execution by the machine, andincludes digital or analog communications signals or other intangiblemedium to facilitate communication of such software.

Although an overview of the inventive subject matter has been describedwith reference to specific example embodiments, various modificationsand changes may be made to these embodiments without departing from thebroader spirit and scope of embodiments of the present application. Suchembodiments of the inventive subject matter may be referred to herein,individually or collectively, by the term “invention” merely forconvenience and without intending to voluntarily limit the scope of thisapplication to any single invention or inventive concept if more thanone is, in fact, disclosed.

The embodiments illustrated herein are described in sufficient detail toenable those skilled in the art to practice the teachings disclosed.Other embodiments may be used and derived there from, such thatstructural and logical substitutions and changes may be made withoutdeparting from the scope of this disclosure. The Detailed Description,therefore, may be not to be taken in a limiting sense, and the scope ofvarious embodiments may be defined only by the appended claims, alongwith the full range of equivalents to which such claims are entitled.

Moreover, plural instances may be provided for resources, operations, orstructures described herein as a single instance. Additionally,boundaries between various resources, operations, modules, engines, anddata stores are somewhat arbitrary, and particular operations areillustrated in a context of specific illustrative configurations. Otherallocations of functionality are envisioned and may fall within a scopeof various embodiments of the present application. In general,structures and functionality presented as separate resources in theexample configurations may be implemented as a combined structure orresource. Similarly, structures and functionality presented as a singleresource may be implemented as separate resources. These and othervariations, modifications, additions, and improvements fall within ascope of embodiments of the present application as represented by theappended claims. The specification and drawings are, accordingly, to beregarded in an illustrative rather than a restrictive sense.

What is claimed is:
 1. A method for reconciling detailed transactionfeedback comprising: detecting, by one or more computer processors, arating of a transaction, the rating classified as indicating a negativeexperience; mining the sentiment of words in feedback text that isincluded with or as part of the rating to detect whether the wordsindicate positive sentiment or negative sentiment; responsive todetermining that the words in the feedback text indicate that the textindicates a positive sentiment, adjusting the rating of the transaction.2. The method of claim 1 wherein mining the sentiment comprises: testingwords in feedback text that is included with or as part of the rating todetect whether the words indicate positive sentiment or negativesentiment; determining whether the tested words in the feedback textindicate a positive sentiment score; and responsive to determining thatthe words in the feedback text indicate that the text indicates apositive sentiment score, adjusting the rating of the transaction. 3.The method of claim 1 wherein adjusting the rating of the transactioncomprises conducting a clarification survey.
 4. The method of claim 1wherein adjusting the rating of the transaction comprises determiningthe average rating of a user for a period of time.
 5. The method ofclaim 2 wherein testing words in feedback text comprises detecting thata word is a negative modifier and, responsive to detecting that a wordis a negative modifier, flagging the feedback text as indicatingnegative sentiment.
 6. The method of claim 2 wherein determining apositive sentiment score comprises totaling the positive words and thenegative words in the feedback text.
 7. The method of claim 1 whereinthe rating of the transaction comprises a numerical rating.
 8. Amachine-readable storage device having embedded therein a set ofinstructions which, when executed by a machine, causes execution of thefollowing operations: detecting a rating of a transaction, the ratingclassified as indicating a negative experience; mining the sentiment ofwords in feedback text that is included with or as part of the rating todetect whether the words indicate positive sentiment or negativesentiment; responsive to determining that the words in the feedback textindicate that the text indicates a positive sentiment, adjusting therating of the transaction.
 9. The machine-readable storage device ofclaim 8 wherein mining the sentiment comprises: testing words infeedback text that is included with or as part of the rating to detectwhether the words indicate positive sentiment or negative sentiment;determining whether the tested words in the feedback text indicate apositive sentiment score; and responsive to determining that the wordsin the feedback text indicate that the text indicates a positivesentiment score, adjusting the rating of the transaction.
 10. Themachine-readable storage device of claim 8 wherein adjusting the ratingof the transaction comprises conducting a clarification survey.
 11. Themachine-readable storage device of claim 8 wherein adjusting the ratingof the transaction comprises determining the average rating of a userfor a period of time.
 12. The machine-readable storage device of claim 9wherein testing words in feedback text comprises detecting that a wordis a negative modifier and, responsive to detecting that a word is anegative modifier, flagging the feedback text as indicating negativesentiment.
 13. The machine-readable storage device of claim 9 whereindetermining a positive sentiment score comprises totaling the positivewords and the negative words in the feedback text.
 14. Themachine-readable storage device of claim 8 wherein the rating of thetransaction comprises a numerical rating.
 15. A system for reconcilingdetailed transaction feedback comprising: one or more computerprocessors and computer storage configured to detect a rating of atransaction, the rating classified as indicating a negative experience;mine the sentiment of words in feedback text that is included with or aspart of the rating to detect whether the words indicate positivesentiment or negative sentiment; responsive to determining that thewords in the feedback text indicate that the text indicates a positivesentiment, adjust the rating of the transaction.
 16. The method of claim15 wherein mining the sentiment comprises: testing words in feedbacktext that is included with or as part of the rating to detect whetherthe words indicate positive sentiment or negative sentiment; determiningwhether the tested words in the feedback text indicate a positivesentiment score; and responsive to determining that the words in thefeedback text indicate that the text indicates a positive sentimentscore, adjusting the rating of the transaction.
 17. The method of claim15 wherein adjusting the rating of the transaction comprises conductinga clarification survey.
 18. The method of claim 1 wherein adjusting therating of the transaction comprises determining the average rating of auser for a period of time.
 19. The method of claim 16 wherein testingwords in feedback text comprises detecting that a word is a negativemodifier and, responsive to detecting that a word is a negativemodifier, flagging the feedback text as indicating negative sentiment.20. The method of claim 16 wherein determining a positive sentimentscore comprises totaling the positive words and the negative words inthe feedback text.